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ABSTRACT 

The effects of providing immediate knowledge of 
results (KR) and adaptive testing on test anxiety and test-taking 
motivation were investigated. Also studied was the accuracy of 
student perceptions of the difficulty of adaptive and conventional 
tests administered with or without immediate knowledge of results. 
Testees were 350 college students divided into high- and low-ability 
groups and randomly assigned to one of four test strategies by KR 
conditions. The ability level of examinees was found to be related to 
their reported levels of motivation and to differences in reported 
motivation under the different testing conditions. These results 
suggest that adaptive testing creates a psychological environment for 
testing which is more equivalently motivating for examinees of all 
ability levels and results in a greater standardization of the 
test-taking environment^ than does conventional testing. (Author) 
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the different testing conditions. Low-ablllty examinees reported slgnlfl- . 
cantly higher levels of motivation on the stradaptlve test than on the 
conventional test, while the reported motivation of hlgh-ablllty examinees 
did not differ as a function of testing strategies. The effect of knowledge 
of results on reported motivation also differed as a function of ability 
level. Low-ablllty testees reported lower motivation with KR than without KR, 
while higher ability testees reported higher motivation v/lth KR. Analysis of 
the anxiety data Indicated that students reported significantly higher levels 
of anxiety on the stradaptlve test than on the conventional test. The 
provision of KR did not result In significant differences In reported anxiety. 
However, highest levels of anxiety were reported by the low-ablllty group on the 
stradaptlve test administered with KR. These results;, In conjunction with 
previously reported data on effects of KR on ability test performance, were 
Interpreted as being the result of facllltatlve anxiety. Students were able to 
perceive the relative difficulty of test Items with some accuracy. However, 
perceptions of the relative degree of test difficulty were much more closely 
related to actual test score on the conventional test than on the stradaptlve 
test. Over 90% of the students reacted favorably to the provision of 
immediate KR. These results suggest that adaptive testing creates a psycho- 
logical environment for testing which is more, .equivalently motivating for 
examinees of all ability levels and results in a greater standardization of the 
test-taking environment, than does conventional testing. 
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Psychological Effects 
OF Immediate Knowledge of Results 
AND Adaptive Ability Testing 

Computer-assisted testing procedures have made it possible to explore and 
implement important new approaches to psychological measurement which have not 
been feasible using the procedures of paper-and-pencil testing. The focus of 
most research to date on adaptive testing (Weiss & Betz, 1973) or tailored 
testing (Lord, 1970) has been on its potential to improve the psychometric 
characteristics of test scores* For example, adaptive and conventional testing 
strategies have been compared on the basis of such psychometric criteria as 
parallel-forms reliability (e.g., Betz & Weiss, 1974, 1975), test-retest 
stability (e.g., Betz & Weiss, 1973, 1975; Larkin & Weiss, 1974; Vale & 
Weiss, 1975a), information curves (e.g.. Lord, 1971a, b; Betz & Weiss, 1974, 
1975; Vale & Weiss, 1975b; McBride & Weiss, 1976), and fidelity or "validity" 
correlations (e.g., Betz & Weiss, 1974, 1975; Vale & Weiss, 1975b; McBride 
& Weiss, 1976) . 

There has been little research, however, on the potential of computer- 
administered testing procedures to improve the psychological environment of 
ability testing. Constructors and users of ability tests should be concerned 
not only with the psychometric properties of tests, but also with the psycho- 
logical effects of those tests on the person being measured, since adverse psy- 
chological effects may reduce the usefulness of a given test score. For example, 
unless people are motivated to do their best on a test, their scores will not 
reflect their maximum performance capabilities. Individuals may become 
anxious, discouraged, or frustrated in testing situations; such reactions may 
interfere with the extent to which test scores reflect those individuals' 
full capabilities. Thus, testing conditions which elicit high levels of test- 
taking motivation and minimize the occurrence of interfering responses may 
be as important as the psychometric properties of the test itself in obtaining 
accurate measurements of individuals. 

The Psychological Environment of Ability Testing 

Test difficulty . It has been suggested (e.g., Hansen, Johnson, Fagan, 
Tam & Dick, 1974; Weiss & Betz, 1973) that adaptive testing procedures should 
create a more favorable psychological environment for all testees than do 
conventional, non-adaptive tests. In the typical conventional testing situation, 
where items are appropriate only for individuals of average ability in a 
defined group, the experience of test-taking is likely to be quite different 
for examinees of different ability levels. Low-ability individuals receive 
items which are often difficult for them and may react by feeling threatened, 
anxious, or frustrated; the test may seem hopeless and they may simply stop 
trying. High-ability individuals, on the other hand, receive items which are 
too easy for them; they may find the task boring and unchallenging and, in a 
fashion similar to that of the low-ability examinees, may simply stop trying 
to do their best. It is only for average-ability examinees that items are 
likely to be sufficiently difficult to be challenging and yet not so diffi- 
cult as to seem hopeless. 
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Adaptive testing procedures, on the other hand, tend to maintain an 
appropriate level of Item difficulty for each Individual, unlike conventional 
tests which do so only for Individuals of average ability. Consequently, the 
motivation of hlgh-ablllty testees may be maintained by the degree of challenge 
presented by Items appropriate to their ability levels. Further, and perhaps 
more Important, low-ability testees may experience less anxiety and frustration 
In an adaptive test than In a conventional test. Consequently, low-ablllty 
testees may be encouraged or motivated to do their best when the Items 
presented to them are near the level of their capabilities. 

Because adaptive testing requires that subsequent test Items be selected 
on the basis of Information derived from analysis of a testee*s responses to 
previous test items, most adaptive testing strategies require computer 
administration. Only a few adaptive testing strategies — the pyramidal test 
(Bayroff, Thomas & Anderson, 1960), the two-stage test (e.g*, Angoff & 
Huddleston, 1958), and the flexilevel test (Lord, 1971c) — have been administered 
in paper-and-pencil mode. However, in the studies in which the pyramidal 
and flexilevel tests were administered in a paper-and-pencil format (Olivier, 
1974; Seeley, Morton & Anderson, 1962), significant numbers of examinees 
invalidated their test protocols by failing to follow the branching instructions 
properly. Thus, even adaptive testing strategies which are designed for 
paper-and-pencil administration can be administered more effectively by 
interactive computers. 

Immediate knowledge of results . Another approach to Improving the 
psychological environment of testing involves telling testees Immediately 
whether each item response was corr«ict or incorrect. Bayroff (1964), 
Ferguson and Hsu (1971) , and others have postulated that Immediate knowledge 
of results has an incentive or motivational effect on examinee performance 
on ability tests. 

While immediate knowledge of results, or KR, can be provided on paper- 
and-pencil tests (e.g., Locke, Cartledge, & Koeppel, 1968; Annett, 1969), its 
provision is inefficient and unwieldly at best. The most frequently used KR 
device, the punchboard developed by Pressey (1950) , is awkward to use and 
requires more effort from examinees who receive KR than from those who do not 
receive KR. Whether because of the difficulty of providing KR on paper-and- 
pencil tests or because of the scarcity of research concerning its effects on 
ability test performance, knowledge of results only rarely has been incor- 
porated into ability testing situations. : 

The administration of ability tests by computer, on the other hand, 
allows fast and efficient provision of KR to examinees without requiring 
additional effort of them. Thus, it is now feasible to investigate the 
effects of incorporating immediate KR as a standard procedure in ability testing. 

Most studies to date on the effects of immediate KR on test performance 
have used classroom achievement tests in which KR was provided using punch- 
board devices or specially constructed answer sheets. Consequently, the 
generalizability of the findings to the situation of computer-administered 
ability tests and computer-administered KR is subject to question. These 
studies, reviewed in detail by Betz & Weiss (1976) , have yielded conflicting 
results. Some investigators have reported that KR enhanced test performance. 
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some that KR led to an increase in errors, and still others that KR conditions 
have no effect on test scores. 

In two studies (Betz, 1975; Sweet & Ringness, 1971) interactions were 
found between the effects of KR and racial or socioeconomic variables. Sweet 
and Ringness (1971) concluded that middle-class students, who typically have 
done relatively well on ability tests, already are highly motivated to do 
well and do not profit from the additional incentive effects of immediate 
KR. Lower-class students, or those who typically show relatively poor 
performance on ability tests, do not tend to be highly motivated in general; 
consequently. Immediate KR may increase their motivation to do well, and 
in turn, increase their test scores. 

An alternative hypothesis concerning the relationship between ability level 
(or "typical test performance level") and the effects of KR was offered by 
Betz and Weiss (1976). According to this hypothesis, high-ability students 
should receive high proportions of positive (i.e., "correct") KR on 
conventional tests constructed to be most appropriate for the average ability 
level in the group. Consequently, KR is likely to encourage, and perhaps thus 
motivate, high-ability examinees. Low-ability examinees, on the other hand, 
receive mostly negative (i.e., "incorrect") KR and may be discouraged, rather 
than motivated. When they receive KR. It is possible, therefore, that the 
quality of KR, or the proportion of positive KR, may be related to the effects 
o£ KR on performance and on the examinee's reactions to KR during testing. 
Thus, in studies of effects of both KR and test difficulty on test performance 
and behavior, the ability level of the examinee may be Important in determining 
the effects of the rt^^ting environment on the psychological reactions of 
examinees, and should be explicitly incorporated into studies of such effects. 

Some investigators (e.g., Ammons, 1956; Ross, 1933) have hypothesized 
that providing immediate knowledge of results to testees may be unnecessary 
because they are always receiving some subjeqtive knowledge of results. 
That is, examinees may have a subjective sense of how well they are performing 
on an ability test. However, there is little data concerning this hypothesis. 
In the only study available (Ross, 1933), the correlation between examinees' 
estimates of their test scores and their actual scores was 2»".71. This 
finding is only tangentially relevant to the effects of KR on test perfor- 
mance, however, since the accuracy of testees' estimates of total scores 
might be substantially different from the accuracy of their perceptions of 
the difficulty levels of a set of test items. 

In suimnary, while the psychological reactions of examinees to testing 
situations may affect the accuracy with which tests can measure their 
abilities, very little is known about those reactions. Further, there is 
little information concerning either the effects of situational variables 
on the psychological reactions of examinees or the relationship between the 
testees' ability level and the extent to which they are psychologically 
able to demonstrate their full capabilities. 
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Objectives 



The purpose of this study was to investigate the psychological reactions 
of examinees to a computer-based ability testing environment incorporating both 
adaptive testing and the provision of immediate knowledge of results. The 
psychological reactions of examinees were considered important because they 
can either facilitate or interfere with the extent to which test scores re- 
flect the "maximum performance" capabilities (Cronbach, 1970) of each indi- 
vidual . 

Within this broad area of interest, there were three specific purposes of 
the investigation. Of central interest were the effects of adaptive versus 
conventional testing and of the provision of immediate knowledge of results on 
the test-taking motivation and test anxiety of high- and low^ability examinees. 
In the study reported by Betz and Weiss (1976), both high- and low-ability 
examinees obtained significantly higher test scores under KR conditions than 
under no-KR conditions, and low-ability examinees obtained higher scores on an 
adaptive test than on a conventional test. Thus, it was of interest to deter- 
mine whether or not the effects of KR and adaptive testing on test performance 
were accompanied by consistent effects of these variables on motivation and 
anxiety. 

A second purpose of the study was to investigate the nature and accuracy 
of subjective knowledge of results, or the examinees* perceptions of test 
difficulty. Subjective KR has been used to explain the absence of effects in 
studies providing objective KR to examinees, but there is little data to sup- 
port or reject its existence as an explanatory construct. 

Further, the existence of accurate perceptions of test difficulty is basic 
to the hypothesis that adaptive testing may improve the psychological environ- 
ment Ox testing, particularly for examinees for whom conventional ability tests 
are inappropriate in difficulty level. For example, it is hypothesized that 
less frustration, discouragement, and anxiety will be induced in low-ability 
examinees when test items administered are appropriately difficult than when 
items are far too difficult for them. This hypothesis is based on the assump- 
tion that low-ability examinees will accurately pevoeive the difficulties of 
the test items. Thus, claims for the psychological benefits of adaptive test- 
ing necessitate demonstration of the existence of accurate subjective KR. 

The final purpose of the study was to investigate examinees' reactions to 
the provision of Immediate KR and the relationship between these reactions and 
the quality of the KR (i.e., the proportion of KR which was positlV^. Since 
Immediate KR had been shown to enhance test performance (Betz & Weiss, 1976), 
data indicating that it also improved the psychological environment of testing 
might suggest the wider inclusion of immediate knowledge of results into test- 
ing procedures* 



METHOD 

Design 

The data were part of a larger study (Betz, 1976; Betz & Weiss, 1976) 
which was concerned with the effects of Immediate knowledge of results and 
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adaptive versus conventional testing strategies tm several aspects of ability 
test performance and examinee behavior. The desig:^^ of the study involved com- 
puterized administration of eifchfer a 50-item conventional ability test or a 
stradaptive ability test (Weiss, 1973), either with or without immediate knowl- 
edge of results, to two groups of subjects. 

One group, consisting of students fr^Dm the College of Liberal Arts at the 
university of Minnesota, was considered the High-Ability group because these 
individuals typically have performed relatively well on ability and scholastic 
aptitude tests. The second group, consisting of students from the University's 
General College, was considered the LM- Ability group because their average 
level of performance on ability and aptitude tests was lower than that of the 
former group. Within each subject group, students were assigned at random to 
one of the four treatment conditions (i.e., the conventional or the stradaptive 
test with or without KR) . 

The major dependent variables of interest in the present report were the 
psychological reactions of the examinees. Immediately after taking the exper- 
imental test, examinees were asked to respond to several Likert-type questions 
indicating how motivated they had been, how anxious they had felt, and how 
difficult the test had seemed to them. From the responses to these questions, 
scales of overall motivation and overall anxiety were constructed. The ques- 
tions concerning test difficulty were used to assess the nature and accuracy 
of examinees' subjective knowledge of results. In addition, those examinees 
receiving immediate KR were asked to respond to questions concerning their 
reactions to its provision. 

Ability Tests 

Item pool . The item pool used to construct the conventional and stradap- 
tive tests of verbal ability consisted of five-alternative multiple-choice 
vocabulary items. The items were calibrated in samples of college undergrad- 
uates, and normal ogive difficulty and discrimination parameters were available 
for each item. Details concerning the development and calibration of the item 
pool are reported in McBride and Weiss (1974). 

Conventional test . The conventional test consisted of 50 items peaked 
around the mean ability level of the high-ability group. In terms of the nor- 
mal ogive item characteristic curve model (Lord & Novick, 1968) the mean 
difficulty level of the 50 items in the test was b=-.20, and the mean discrimi- 
nation value was a=.89. Further details concerning the characteristics and 
scoring of the peaked conventional test may be found in Betz and Weiss (1976, 
p. 12). 

Stradaptive test . In constructing the stradaptive test, the items in the 
pool were grouped into nine levels, or strata, on the basis of their difficulty. 
(For details of the constjruction of the stradaptive test see Betz and Weiss, 
1976, p. 10). There was no overlap in item difficulty between adjacent strata, 
and the difficulty ranges of the items spanned the difficulty continuum from 
b=-3 to b«+3. Mean difficulties of the strata ranged from F«-2.67 to 5=2.63. 
Once items had been grouped into difficulty levels, they were ordered by dis- 
criminating power. There were a maximum of 30 items in a stratum, and all 
items had a minimum discrimination of a«.30 (corresponding to an item-total 
score biserial correlation of .28). 
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Examinees began the stradaptive test in a stratum determined from their 
reported grade-point averages (GPA) ; those examinees reporting high CPAs 
began the test with more difficult items than did those reporting lower CPAs. 
Examinees were branched through the stradaptive item structure according to 
the rule tha*t following a correct response, the most discriminating item re- 
maining in the next more difficult stratum was administered, and following an 
incorrect response, the most discriminating item in the next less difficult 
stratum was administered. Testing was continued in this manner until either 
a ceiling stratum (see Weiss, 1973, p. 17-20; Betz & Weiss, 1976, p. 10) had 
been identified or 75 items had been administered. 

All-Item ability estimate . In order to study the relationships between 
psychological reactions to testing and ability level, a single maximum like- 
lihood ability estimate (Betz & Weiss, 1976, p. 11) was calculated from each 
examinee's responses to the items in the experimental test and to the 44«-item 
post-test administered as the last part of the experimental procedure. The 
post-test was a conventional computer-administered test constructed using 
vocabulary items from Educational Testing Service's Cooperative School and 
College Ability Tests, forms 2A, 2B, 3A, and SB (see Betz & Weiss, 1976, 
p. 12, for details of its construction). The items in the post-test, like 
those in the item pool used for the experimental tests, used a five-alterna- 
tive multiple-choice format. The post-test was administered without KR. Thus, 
the All-Item ability estimate was based on all of the item response data 
available for each individual. It included responses to either the peaked 
conventional or the stradaptive test, administered with or without KR, and 
responses to the 44-item post-test. 

Psychological Reactions 

Since very little research, other than that on state anxiety and test 
anxiety, has concerned itself with affective reactions toward testing situa- 
tions, there were few guidelines as to how to assess the psychological effects 
of testing. Several types of examinee reactions were of interest in the 
present study: 1) level of reported motivation to do well; 2) level of state 
anxiety; 3) perceived difficulty of test items and of the test as a whole in 
relationship to the examinee's ability level (i.e., the presence and accuracy 
of subjective knowledge of results); and 4) reactions to the provision of KR. 
Because of the variety of types of information desired from the examinee, it 
was possible to include only a few questions designed to measure each type of 
reaction. Questions measuring the various psychological reactions were inter- 
mixed and administered immediately following the experimental test, but prior 
to the post-test* 

Motivation . Four items were written to assess level of examinee motiva- 
tion. Table 1 shows these items and the serial position in which they were 
administered^. Two items, "Did you feel challenged to do as well as you could 



^In addition to questions concerned with motivation, anxiety, perception of 
test difficulty and reactions to KR, six additional questions (Items 5, 8, 
14, 16, 17, 23) were asked of the testees. These items were primarily con- 
cerned with guessing behavior and were not analyzed for this report. 

11 



7 



on the test?" and "Did you care how well you did on the test?" reflected a 
general motivation to do well. The item "How frequently were you careful to 
select what you thought was the best answer to each question?" was intended 
to operationalize level of motivation in terms of the behavior of careful 
versus careless responding. The fourth item "Do you think that you could have 
done better if you had tried harder?" indicated the extent to which examinees 
felt they had tried to demonstrate their maximum capabilities. 



Table 1 

Motivation Items, Serial Position of Administration, 
and Weights Assigned to Response Alternatives 



Item and Response Serial Assigned 

Alternatives Position Weight 



How frequently were you careful to 
select what you thought was the 

best answer to each question? 6 

1. Almost always 4*0 

2. Frequently 3-2 

3. Sometimes 2,4 

4. Rarely 1-6 

5. Never -8 
Do you think that you could have done 

better on the test if you had tried harder? 9 

1. I definitely could have -8 

2. I probably could have 1-6 

3. I*m not sure 2,4 

4. I probably couldn't have 3-2 

5. I definitely couldn't have 4.0 
Did you feel challenged to do as well 

as you could on the test? 13 

1. Not at all 1 

2. Somewhat 2 
. 3. Fairly much so 3 

4. Very much so 4 
Did you care how well you did on the test? 18 

1. I cared a lot 4.0 

2. I cared some 3.2 

3. I cared a little 2.4 

4. I cared very little \ 1-6 

5. I didn't care at all ^--^ — .8 



Anxiety , Three items were included to assess level of reported anxiety. 
These items and their response alternatives are shown in Table 2, which also 
indicates the serial position of each item within the series of reactions 
items administered. Two of the items, "Were you nervous while taking the test?" 
and "How did you feel while taking the test?" were patterned after the Items 
used by Hedl (1973) in his five-item version of the A-State Scale from the 
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State-Trait Anxiety Scale (Spielberger, Gorsuch & Lushene, 1969). The third 
item, "During testing, did you worry about how well you would do?" is similar 
to some items used in the Test Anxiety Questionnaire (TAQ; Handler & Sarason, 
1952) and the Test Anxiety Scale (Sarason, 1958) , which was derived from items 
in the TAQ. 

Table 2 

Anxiety Items, Serial Position, and Weights Assigned 
to Response Alternatives 

Item and Response Serial Assigned 

Alternatives Position, Weight 



During testing; did you. worry 

about how well you would do? 4 

1. Not at all 

2. Somewhat 

3. Fairly much so 

4. Very much 

Were you nervous while taking the test? 7 

1. Not at all 

2. Somewhat 

3. Moderately so 

4. Very much so 

How did you feel while taking the test? 11 

1. Very tense 

2. Somewhat tense 

3. Neither tense nor relaxed 

4. Somewhat relaxed 

5. Very relaxed 



1 
2 
3 
4 

1 
2 
3 
4 

4 

3.2 
2.4 
1.6 
.8 



Quantification of motivation and anxiety levels . Motivation and anxiety 
scales were constructed by coinbining responses to the individual motivation 
and anxiety items. In constructing the scales, each of the constitutent items 
was given equal weight in determining a total score. Accordingly, since the 
items used either four or five response alternatives, responses to items 
having five alternatives were reweighted to a scale of I to 4. Items also were 
reweighted so that the response alternative indicating highest level of moti- 
vation or most anxiety would receive a weight of 4; alternatives Indicating 
least motivation or anxiety received a weight. of I for a four-response item, 
or .8 for a five-response item. Tables I and 2 contain the weights assigned 
to each response alternative in calculating t:otal motivation scores and total 
anxiety scores, respectively. Total scores were obtained by adding the iieights 
corresponding to the chosen alternative for each of the Items In the scale. 

Perceived test difficulty . Six Items were written to assess the nature 
and accuracy of perception of test difficulty; these Items are shown In > 
Table 3. Four of the Items, including **How oftei^ did you feel that the ques- 
tions in the test were too easy for you?" and "In relation to your vocabulary 
ability, how difficult was the test for you?" reflected examinees' perceptions ,| 
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of the degree of difficulty of the test questions in relationship to their 
knowledge or ability. The other two items were concerned with the effects 
(e.g., frustration) of the perceived difficulty of the test on the testee's 
performance levels. 



Table 3 

Perception of Test Difficulty Icexns 
and Serial Position of Administration 



Item and Response ^^®5^^^ 
Alternatives 

How often did you feel that the questions in the 

test were too easy for you? ^ 

1. Always 

2. Frequently 

3. Sometimes 

4. Seldom 

5. Never 

How often did you feel that the questions in the 

test were too hard for you? ^ 

1. Always 

2. Frequently 

3. Sometimes 
4« Seldom 

5. Never 

How often did you feel that the questions in this 

test were just about right for someone of your ability? 3 

1. Always 

2. Frequently ^ — - 
3* Sometimes 

4. Seldom \ 

5. Never 

In relation to your vocabulary ability, how difficult 
was the test for you? 

1. Much too difficult 

2. Somewhat too difficult 

3. Just about right 

4. Somewhat too easy 
5* Much too easy 

Did you feel frustrated by the difficulty of the 
test questions? 

1. Not at all 

2. Somewhat 

3. Fairly much so 

4. Very much so 

Hov veil do you feel you did on this test In 

comparison to your performance on other tests like this? 15 

1. Much better 

2. Somewhat better 
3* About the same 

4. Somewhat worse 

5. Much worse 



Reactions to KR > Those examinees receiving KR were aaked to respond to 
seven items concerning their reactions to its provision; these items are shown 
in Table 4. In these items, the term "feedback" was used instead of the term 
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"KR" since It was thought that students would be more familiar with the former 
term than with the latter. Three Items, "Did getting feedback after each 
question make you nervous?", "Did getting feedback after each question inter- 
fere with your ability to concentrate on the test?", and "How did you feel 



Table 4 

Items Concerning Reactions Co the Provision o£ 
KR and Serial Position o£ Administration 



Item and Response 

Alternatives Serial Position 

Did receiving feedback a£ter each question Interfere with 

your ability to concentrate on the test? 20 

1. Mo, not at all 

2. Yes, somewhat 

3. Yes, moderately so 

4. Yes, very much so 

Did getting feedback after each question make you nervous? 21 

1. Mo, not at all 

2. Yes, somewhat 

3. Yes, moderately so 

4. Yes, very much so 

How did you feel when you found that your answers were Incorrect? 25 

1. It bothered me a lot 

2. It bothered me some 

3. It bothered me a little 

4. It didn't bother me at all 

Did getting feedback on this test make It more Interesting 

or less Interesting? 19 
1« Much more interesting 

2. Somewhat more Interesting 

3. Didn't make any difference 

4. Somewhat' less Interesting 

5. Much less Interesting 

Did you try harder to get the questions right because you knew 

you would get feedback after each question? 22 

1. Mo, not at all 

2. Yes, somewhat 

3« Yes, moderately so 
4. Yea, very much so 
Vare you Interested In knowing whether your answers were right 

or wrong? 24 

1. I was very interested 

2. I was moderately Interested 

3. I was somewhat Interested 

4. I didn't care at all 

How do you feel about getting feedback? > 26 

1. I'd rather not know whether my answers were right or wrong 

2. I really don't care whether I get feedback or not 

3. I liked getting the feedback 



when you found that your answers were Incorrect?" were included to determine 
whether providing KR had any potentially disruptive effects on examinee per- 
formance. The item "Did you try harder to get the questions right because you 
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knew you would get feedback after each question?" reflected the relationship 
between KR and motivation to do well. The other three items, including "Did 
getting feedback on this test make it more or less interesting?", assessed 
the extent to which examinees felt that receiving KR had made test-taking a 
more or less interesting and/or positive experience for them. 



Procedures 

The data for this study were obtained from the subjects and experimental 
procedures reported by Betz (1976) and Betz and Weiss (1976). Briefly, each 
of 350 students (239 High-Ability and 111 Low-Ability) completed a computer- 
administered experimental ability test, the series of psychological reactions 
items, and a post-test. The experimental test, assigned at random to students 
within ability groups, was either a conventional or stradaptive test adminis- 
tered with or without immediate knowledge of results. All students were tested 
individually at cathode-ray terminals connected to a real-time minicomputer. 

Analysis of Data 

Data of interest in this study consisted of the following for each 
examinee: 1) responses to the individual motivation, anxiety, and perception 
of difficulty items; 2) scores on the composite anxiety and motivation scales; 
and 3) All-Item maximum likelihood ability estimates. Additional da^a: : ^%il- 
able for examinees who had received KR included their reactions to the 
provision of KR and the proportions of positive and negative KR received (ob- 
tained from calculation of the percentage of correct responses to the Items in 
the experimental test) . 

Motivation and anxiety . Mean motivation and anxiety composite scores were 
analyzed using a "classic experimental" three-way analysis of variance (Nie, 
Hull, Jenkins, Steinbrenner & Bent, 1975, pp. 405-408). KR, testing strategy 
and ability level group were the independent variables. The mean and standard 
deviation of scores within each treatment-subject group combination were cal- 
culated, and post-hoc comparisons of sub-group means were made using Scheff^'s 
(1959) method. In addition, product-moment correlations were calculated 
between composite motivation scores and ability level, as determined by the 
All-Item ability estimate, and between composite anxiety scores and ability 
level. 

In order to examine students* psychological reactions at the item level, 
distributions were obtained of the percentages of individuals selecting each 
response alternative of the motivatibti and anxiety Items, both for the total 
group of examinees and as a function of KR conditions and testing strategy. 
Chi-square tests of independence were used to determine whether there were 
significant relationships between item responses and KR conditions or between 
item responses and testing strategy. 

Perceived test difficulty . The percentage of individuals selecting each 
response category of the perception of test difficulty items was obtained for 
the total group and as a function of KR and testing strategy. To determine the 
accuracy of perceived test difficulty, correlations were computed between 
scores on each of the six Items (using arbitrary 1 through 5 weights) and the 
All-Item ability estimate. Finally, in order to examine the effects of KR and 
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testing strategy on accuracy of perceived test difficulty, correlations were 
calculated between the item responses and the All-Item ability score within 
the two KR treatment groups and within the groups receiving different experi- 
mental tests. 

Reactions to the provisicn of KR , Analysis of reactions to the provision 
of KR was intended primarily for descriptive purposes, since little J.s known 
about how individuals react to it. Accordingly, percentages of examinees 
selecting each response category and the mean and standard deviation of item 
scale values were calculated. 

Additional analyses were concerned with whether or not the proportion of 
positive KR Influenced the reactions of examinees to its provision. In these 
analyses the percentage of correct ability test item responses was calculated 
for each person. Correlations then were calculated between percentage correct 
and response? to the KR items, assigning arbitrary integer weights to the item 
responses. The mean and standard deviation of the percentage correct scores 
were calculated within the conventional and stradaptive test groups to deter- 
mine the relative percentages of positive KR for the two testing strategies. 



RESULTS 

Motivation 

Mean motivation scores . Table 5 presents the means and standard 
deviations of the motivation composite scores as a function of Ability 
group, testing strategy, and KR. Also shown are results of the three-way 
analysis of variance. There was a significant main effect for Ability group, 
and there were significant interactions between Ability group and testing 
strategy, and between Ability group and KR. In general, the High-Ability 
group reported significantly higher levels of motivation than did the Low- 
Ability group. This significant group difference reflected the higher levels 
of motivation for the High-Ability group on both tests, as compared to the high 
levels of laotivation for the Low-Ability group only on the stradaptive test. 
The mean level of motivation reported by Low-Ability students taking the 
stradaptive test (12.5) was significantly higher than that reported by Low- 
Ability students completing a conventional test (11.3). In contrast, the means 
for High-Ability students taking the stradaptive test (12.4)' and the 
conventional test (12.6) were not significantly different. While High-Ability 
students reported significantly higher levels of motivation on the conventional 
test than did Low-Ability students, on the stradaptive test there was no 
significant difference in reported motivation between the two groups. 

The interaction between group and knowledge of results reflected the 
different reactions of the High- and Low-Ability groups to the provision of 
KR. In the High-Ability group, mean levels of reported motivation were 
slightly higher under KR conditions (12.6) than under No-KR conditions 
(12.3). In the Low-Ability group, motivation was lower under KR conditions 
(11.5) than under No-KR conditions (12.2). While this latter difference was 
not statistically significant, it was larger and in the opposite direction 
than the difference between KR and No-KR conditions in the High-Ability group, 
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indicating that KR had a different type of influence on the levels of 
motivation reported by. High- and Low-Ability examinees. 

Table 5 

Means and Standard Deviations of Motivation Scores 
for Conventional and Stradaptive Tests in High- and 
Low-Ability Groups With and Without KR 



Test and Group 



Combined 





KR 






No-KR 




Conditions 


N 


Mean 


S.D. 


N 


Mean 


S.D. 


N 


Mean 


S.D. 


60 


12.7 


1.8 


57 


12.4 


2.3 


117 


12.6 


2.02 


28 


11.0 


2.1 


28 


11.5 


2.6 


56 


11.3 


2.38 


60 


12.5 


2.0 


62 


12.1 


2.2 


122 


12.4 


1.78 


28 


12.0 


2.8 


27 


13.0 


1.9 


55 


12.5 


2.45 


88 


12.2 


2.0 


85 


12.2 


2.4 


173 


12.2 


2.2 


88 


12.4 


2.3 


89 


12.5 


1.7 


177 


12.4 


2.0 


120 


12.6 


1.9 


119 


12.3 


2.4 


239 


12.5 


1.9 


56 


11.5 


2.5 


55 


12.2 


2.4 


111 


11.9 


2.5 


176 


12.3 


2.1 


174 


12.3 


2.3 


350 


12.3 


2.2 



Conventional 
High-Ability 
Low- Ability 

Stradaptive 
High-Ability 
Low- Ability 

Combined Groups 
Conventional 
Stradaptive 
High-Ability 
Low-Ability 
Total 



Three-Way Analysis of Variance 



Source of 
Variation 



Sum of 




Mean 


F 




Squares 


DF 


Square 




38.84 


3 


12.95 


3.01 


.030 


31.21 




31.21 


7.25 


.007 


6.91 




6.91 


1.61 


■ .203 


.33 




.33 


.08 


.999 


52.36 




17.45 


4.05 


.008 


33.71 




33.71 


7.83 


.006 


18.04 




18.04 


4.19 


.039 


.75 




.75 


.17 


.999 


.83 




.83 


.19 


.999 


1468.39 


341 


4.31 






1560.42 


348 


4.48 







Main Effects 
Ability Group 
Test 
KR 

Two-Way Interactions 

Ability Group x Test 

Ability Group x KR 

Test X KR 
Three-Way Interaction 

Ability Group x Test x KR 
Residual 
Total 



Estimated probability of error in rejecting null hypothesis. 




Figure 1 shows the means for the eight experimental groups. The dashed 
line in Figure 1 encloses subgroup means that were not significantly different 

rom each other. As ^Figure 1 shows, testing conditions had significant effects 
•^n the reported motivation of Low-Ability students. Means for the Low-Ability 
group who completed the conventional test with KR (11.0) or without KR (11.5) 
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were significantly different from the means for all other treatment groups. 
Thus, the Low-Ability group reported significantly lower motivation to perform 
well when administered a conventional test, with or without KR, 



Figure 1 

Mean Motivation Scores as a Function 
of KR Condition, Testing Strategy, and Ability Group 




KR No-KR 



The data also show that while the High-Ability group reported a higher 
overall level of motivation than the Low-Ability group, levels of motivation 
reported by Low-Ability examinees on the stradaptive test under either KR or 
No-KR conditions were not significantly different from those reported by 



High-Ability examinees under any of the four testing conditions. Testing 
conditions did not have significant effects on the levels of motivation re- 
ported by High-Ability examinees; these individuals appear to be highly moti- 
vated regardless of conditions. 

Responses to motivation items . Table 6 indicates the percentage of 
examinees selecting each alternative of the four motivation items, as a 
function of KR, testing strategy and for the total group. In general, it 
appears that examinees were fairly highly motivated; 57.6% of the total group 
reported that they almost always had tried to select the best answer to each 
question. Most (57%) felt that they "definitely" or "probably" could not have 



Table 6 

Response Percentages for Motivation Items as a Function of 
KR Condition, Testing Strategy, and for Total Group 



Item and Response 
Categories 



KR Condition 
KR No-KR p' 



Testing Strategy 



Conventional Stradaptive p 



Total 
Group 



1. Hov frequently were you careful to 
select what you thought was the 
best answer to ^ach question? 

Almost always 64.1 
Frequently 26 . 1 

Sometimes 10 •B 

Rarely 1-7 
Never 0 

2. Do you think you could have done 
better on the test if you had 
tried harder? 

I definitely could have 5.7 
I probably could have 13*6 
I'm not sure 30-7 
I probably couldn't have 35.8 
I definitely couldn't have 14.2 

3. Did you feel challenged to do as 
well aa you could on the test? 

Not at all ^-5 
Somewhat 29.5 
Fairly much so 36.4 
Very much so 29.5 

4. Did you care how well you did 
on the test? 

I cared a lot 19.9 
I cared some 56.3 
I cared a little 14.8 
I cared very little 6.8 
I didn't care at all 2.3 



53.8 
34.1 
8.1 
2.3 
1.7 



, 4.0. 

6.4 
25.4 
39.9 
24.3 



8.1 
27.2 
37.2 
27.2 



19.1 
50.3 
22.0 
6.9 
1.7 



.16 



.03 



.55 



.52 



60.7 
27.2 
8.7 
1.7 
1.7 



6.4 
12.1 
27.2 
37.0 
17.3 



6.4 
27.7 
40.5 
25.4 



18.5 
49.7 
21.4 
8.1 
2.3 



54.5 
33.0 
10.2 
2.3 
0 



3.4 
8.0 
29.0 
38.6 
21.0 



6.3 
29.0 
33.5 
31.3 



20.5 
56.8 
15.3 
5.7 
1.7 



.29 



.43 



.53 



.46 



57.6 
30.1 
9.5 
2.0 
.9 



4.9 
10.0 
28.1 
37.tf 
19.2 



6.3 
28.4 
37.0 
28.4 



19.5 
53.3 
18.3 
6.9 
2.0 



note. 



Items are numbered 1 to 4 only for purposes of diecusaion. The order in which items were 
administered is contained in Table 1. Nuinber of testees in each condition is shown in Tabla 5. 
^Probability of error in rejecting null hypothesia of independence, based on chi-square test with 
four degrees of freedom for items 1, 2, and 4, and three degrees of freedom for item 3. 



done better if they had tried harder, and 65% of examinees reported being 
either fairly much or very much challenged to do their best. Almost three- 
quarters of the examinees cared "some" or "a lot" about how well they did on 
the- test. 
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There were significant differences in responses to motivation Item 2 
as a function of KR conditions; testees receiving KR were more likely to report 
that they could have done better if they had tried harder than were those not 



Table 7 

Means and Standard Deviations of Anxiety Scores 
for Conventional and Stradaptive Tests in High- and Low-Ability Groups 
With and Without KR, and Results of the Three-Way ANOVA 



Test and Group 



Experimental Condition 



KR 



Mean S • D . 



No-KR 



Mean 



;.D, 



Combined 
Conditions 
Mean S.D, 



Conventional Test 

High-Ability 

Low-Ability 
Stradaptive Test 

High-Ability 

Low- Ability 
Combined Groups 

Conventional Test 

Stradaptive Test 

High-Ability 

Low- Ability 

Total 



5.3 


1.7 


5.5 


1.8 


5.4 


1.6 


5.4 


2.0 


6.0 


2.4 


5.7 


2.2 


6.0 


2.2 


5.8 


1.8 


6.0 


2.0 


6.9 


2.5 


5.9 


2.2 


6.4 


2.4 


5.4 


1.8 


5.6 


1.9 


5.5 


1.8 


6.3 


2.4 


5.9 


1.8 


6.1 


2.1 


5.7 


2.0 


5.6 


1.7 


5.7 


1.8 


6.2 


2.4 


5.9 


2.3 


6.0 


2.3 


5.8 


2.1 


5.7 


1.9 


5.8 


2.0 





Three-Way Analysis 


of Variance 






Source of 


Sum of 


Mean 




a 


Variation 


Squares 


DF Square 


F 


P 



Main Effects 40.48 

Ability Group 10.64 

Test 29.73 

KR -62 

Two-Way Interactions 12.53 

Ability Group x Test .39 

Ability Group x kR 1.10 

Test X KR 11-12 

Three-Way Interaction 

Ability Group x Test x KR 8.20 

Residual 1332.59 

Total 1393.80 



34 
348 



13.49 
10.64 
29.73 
.62 
4.18 
.39 
1.10 
11.12 

8.20 
3.91 
4.01 



3.45 
2.72 
7.61 
.16 
1.07 
.099 
.283 
2.85 

2.10 



.017 

.096 

.006 

.999 

.36 

.999 

.999 

.09 

.14 



Note. Subgroup Ns are shown In Table 5. 

Estimated probability of error in rejecting null hypothesis. 



receiving KR. However, examinees with KR also reported more often than those 
without KR that they had tried to select the best answer to each question, al- 
though this tendency was not statistically significant. 



There were no significant differences in response percentages as a func- 
tion of testing strategy. Examinees taking the conventional and stradaptive 
tests were approximately equally likely to report that they had tried to 
select the best answer to each question, that they felt challenged to do their 
best on the test, that they couldn't have done better if they had tried harder, 
and that they cared how well they did on the test. 

Motivation and ability level . There was a statistically significant 
(p=.001) correlation of r=.21 between overall motivation score and ability 
level measured using the All-Item ability estimate. Thus, it appears that 
higher overall performance and higher motivation are associated, although the 
amount of variance in common is low. All four individual motivation items 
were significantly related to ability level at or below the .05 level of 
statistical significance. The higher the ability levels of examinees, the more 
likely they were to 1) report trying to select the best 'answer to each question, 
2) say that they couldn't have done better if they had tried harder, 3) report 
being challenged to do as well as possible, and 4) care how well they did on 
the test. 

Anxiety 

Mean anxiety scores . Table 7 shows the results of the three-way analysis 
of mean anxiety scores as a function of KR, test, and subject group, and the 
means and standard deviations of anxiety scores for each experimental subgroup 
combination. Figure 2 provides a plot of the eight subgroup means. 

As shown in Table 7, the only significant main effect was for testing 
strategy; the overall level of anxiety reported by individuals taking the 
stradaptive test (6.1) was significantly greater than that reported by indivi- 
duals taking the conventional test (5.5). The main effect of KR was not sta- 
tistically significant; the mean anxiety scores of examinees tested under KR 
(5.8) and No-KR (5.7) conditions were similar. However, the interaction between 
test and KR approached statistical significance (p<.10). Examinees reported 
more anxiety on the stradaptive test with KR (6.3) than without KR (5.9) but 
reported more anxiety on the conventional test without KR (5.6) than with KR 
(5.4). With KR, the difference between the stradaptive test mean of 6.3 and 
conventional test mean of 5.4 was statistically significant, while \-7ithout KR 
the difference (5.9 vs. 5.6) was not significant. 

The main effect for Ability Group also approached statistical significance 
(p<.10). The mean level of reported anxiety for Low-Ability students over all 
conditions (6.0) was slightly higher than that for High-Ability students over 
all conditions (5.7). This difference, however, was due primarily to the 
significantly higher mean level of anxiety (6.9) reported by Low-Ability stu- 
dents taking the stradaptive test with KR, as shown in Figure 2. As Figure 2 
also shows, difference among the means for the remaining seven experimental sub- 
groups were not statistically significant. 

Responses to anxiety items . Table 8 shows the percentage of examinees 
endorsing each alternative of the three anxiety items as a function of KR con- 
ditions, testing strategy and for the total group. In general, the total group 
of students did not report experiencing much anxiety during testing. The modal 
responses to the three questions indicated that they worried about how well 
they would do but did not feel particularly nervous or tense. 
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There were no significant differences In reported anxiety as a function of 
KR; If any trend at all Is present It Is In the third Item, where testees re- 
ceiving KR report feeling tense or somewhat tense more often than do those not 
receiving KR. 



Figure 2 

Mean Anxiety Scores as a Function of KR Condition, 
Testing Strategy, and Ability Group 
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report feeling moderately or somewhat nervous while taking the test (p=.02), 
to report worrying about how well they would do (p-.08), and to report being 
tense, rather than relaxed, while taking the test (p».13) than were examinees 
taking the conventional test. 



Table 8 

Response Percentages for Anxiety Items as a Function of 
KR Condition, Testing Strategy, and for Total Group 



Item and Response 
Categories 



Testing Strategy 



KR Condition 

KR No-KR p* Conventional Stradaptive p** 



Total 
Group 



1. 


During testing did you worry 






.66 






.08 






about hov veil you would do? 










19.2 




Not at all 


21.6 


16.8 




21.4 


17.0 






Somewhat 


53.4 


54.3 




57.8 


50.0 




53.9 




Fairly much 


14.8 


16.8 




11.6 


19.9 




15.8 




Very much 


10.2 


12.1 




9.2 


13.1 




11.2 


2. 


Were you nervous while taking 












.02 






the test? 






.56 






62.2 




Not at all 


59.1 


65.3 




68.8 


55.7 






Somewhat 


28.4 


25.4 




23.7 


30.1 




26.9 




Moderately so 


9.1 


7.5 




4.6 


11.9 




8.3 




Very much so 


3.4 


1.7 




2.9 


2.3 




2.6 


3. 


How did you feel while taking 












.13 






the test? 






.15 






2.3 




Very tense 


3.4 


1.2 




• 6 


4.0 






Somewhat tense 


22.7 


16.2 




17.9 


21.0 




19.5 




Neither tense nor relaxed 


28.4 


38.7 




31.8 


35.2 




33.5 




Somewhat relaxed 


28.4 


27.2 




31.2 


24.4 




27.8 




Very relaxed 


17.0 


16.8 




18.5 


15.3 




16.9 



HoU. Items are numbered 1 to 3 only for purposes of discussion. The order in %rtiich items were 

administered is contained in Table 2. Number of testees in each condition is shown in Table 5. 

*Prob«bility of error in rejecting null hypothesis of independence, based on chi-square test with 
three degrees of freedom for item 1 and four degrees of freedom for items 2 and 3. 



Anxiety and ability level . The correlation between the all-item ability 
estimate and composite anxiety score was -.10. This value of V was significant 
at p=.04. Responses to only one anxiety item showed a significant (p=.005) 
relationship to ability level: High-Ability examinees were le6& likely to re- 
port worrying during testing than were low ability examinees. The slight nega- 
tive correlation between ability level and reported anxiety is in agreement with 
the tendency for the Low-Ability group to report somewhat higher levels of 
anxiety than the High-Ability group. * 

Perceived Test Difficulty 

Responses to test difficulty items . Table 9 presents the percentage of 
examinees selecting each response category of the six items used to measure 
perceived difficulty of the test taken. Results for the total group can be 

summarized as follows: 1) examinees in general felt that the experimental ..^ 

ability test items were "seldom" too e^sy, were "frequently" too hard, and were | 

"sometimes" just about right for their ability level; 2) most examinees felt 

that, as a whole, the experimental test was "somewhat" too difficult for them; 

3) they reported being ''somewhat" frustrated by the difficulty of the test -jj 
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Table 9 

Response Percentages for Subjective Knowledge of Results Items as a Function of 
KR Condition, Testing Strategy, and for Total Group 



Item and Response 
Categories 



KR Condition 
KR No-KR 



Testing Strategy 



Conventional Stradaptive 



Total 
Group 



1. How often did you feel that the 
questions in the test were too 
easy for you? 

Always 

Frequently 

Sometimes 

Seldom 

Never 

2. How often did you feel that the 
questions in the test were too 
difficult for you? 

Always 

Frequently 

Sometimes 

Seldom 

Never 

3. How often did you feel that the 
questions In this test were just 
about right for someome of your 
ability? 

Always 

Frequently 

Sometimes 

Seldom 

Never 

A. In relation to your vocabulary 
ability, how difficult was this 
test for you? 
Much too difficult 
Somewhat too difficult 
Just about right 
Somewhat too easy 
Much too easy 

5. Did you feel frustrated by the 
difficulty of the test questions? 

Not at all 
Somewhat 
Fairly much so 
Very much so 

6. Bow well do you feel you did on 
this test in comparison to your 
performance on other tests like 
this? 

Much better 
Somewhat better 
About the same 
Somewhat worse 
Much worse 



0 

5.7 
36.9 
50.6 

6.8 



4.0 
46.0 
36.4 
11.9 

1.7 



4.5 
30.1 
50.6 
13.1 

1.7 



5.1 
58.0 
31.8 

5.1 
0 



34.7 
54.5 
9.7 
1.1 



1.1 . 
12.5 
51.1 
29.5 

5.7 



1.2 
2.3 
30.6 
51.4 
14.5 



5.2 
53.8 
33.5 
6.4 
1.2 



1.7 
22.0 
52.6 
19.7 

4.0 



15.0 
62.4 
20.2 
1.7 
.6 



26.0 
56.1 
13.9 
4.0 



0 
6.4 
59.4 
27.2 
7.5 



.03 



.32 



.07 



.001 



.09 



.13 



1.2 
5.2 
32.9 
48.6 
12.1 



5.8 
48.0 
32.9 
11.6 

1.7 



4.0 
26.0 
49.1 
16.8 

4.0 



12.7 
54.9 
26.6 
5.2 
.6 



31.8 
56.1 
9.8 
2.3 



.6 
12.7 
57.8 
22.5 
6.4 



0 

2.8 
34.7 
53.1 

9.1 



3.4 
51.7 
36.9 
6.8 
1.1 



2.3 
26.1 
54.0 
15.9 

1.7 



7.4 
65.3 
25.6 

1.7 
0 



29.0 
54.5 
13.6 
2.8 



.6 
6.3 
52.3 
34.1 
6.8 



.34 



.40 



.56 



.08 



.70 



.08 



.6 
4.0 
33.8 
51.0 
10.6 



4.6 
49.9 
35.0 
9.2 
1.4 



3.2 
26.1 
51.6 
16.3 

2.9 



10.0 



60. 
26. 
3 



30.4 
55.3 
11.7 
2.6 



.6 
9.5 
55.0 
28.4 
6.6 



notB IteM are numbered 1 to 6 only for purposes of discuaaion. The order In which items were 

administered is contained in Table 3. Number of testeea in each condition la shown in Table 5. 
■probability of error In rejecting null hypothesis of Independence, based on chl-squar« test with 
four degrees of freedom for items 1 thru 4 and 6, and three degrees of freedom for item 5. 



questions; and 4) they felt that their performance on this test was comparable 
to that on other similar tests. In general, then, the test was perceived as 
somewhat too hard and somewhat frustrating* 
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However, when KR was taken into account there was a significant relation- 
ship between perception of difficulty and knowledge of results. Examinees 
with KR were more likely to consider the questions too easy "frequently" or 
"sometimes" (Item 1) and were more likely to consider the difficulty of the 
test as a whole "just about right" for their ability level (Item 4). In addi- 
tion, on Item 3 examinees with KR showed a marginally significant (g=-07) 
tendency to consider the individual items more appropriate for their ability 
levels and on Item 5 a tendency (p=.09) to report less frustration with the 
difficulty of the test questions. These four items suggest that examinees 
receiving KR perceived the test as a less difficult and frustrating experience 
than did those not receiving KR. 

Although there were no significant (p<.05) differences as a function of 
testing strategy, examinees taking the stradaptive test were somewhat more 
likely to report that the test as a whole was too difficult for them (Item 4, 
p».08) and that their performance on the test was "somewhat worse" than their 
usual levels of performance on such tests (Item 6, p=.08) . 

Accuracy of perceived test difficulty . Table 10 shows the correlations 
between responses to the items measuring perceived test difficulty and the All- 
Item ability estimate for the total group, and as a function of both KR and 
testing strategy. Each of the six items showed a moderate relationship to abil- 
ity level for the total group; the values of v for the total group all were 
significantly different from zero at p<.002. 



Table 10 

Correlations Between Items Measuring 
Perceived Test Difficulty and Ability Level as 
a Function of KR Condition, Testing Strategy, and for Total Group 







KR Condition 


Testing Strategy 


Total 






KR 


No-KR 


Conventional 


Stradaptive 


Group 


a 

Item 




(N=176) 


(N=174) 


(N=173) 


(N=177) 


(N=350) 


1 




-.42 


-.35 


-.58 


-.19 


-.39 


2 




.55 


.40 


.64 


.33 


.48 


3 




-.31 


-.32 


-.44 


-.21 


-.33 


4 




.52 


.42 


.68 


.25 


.47 


5 




-.24 


-.29 


-.47 


-.11 


-.28 


6 




-.23 


-.08 


-.34 


-.00 


-.16 


fJotc. 


For 


a sample size of 173 


, differences 


greater than 


or equal 




to r 


'=.22 between Z-transformed values 


of r are significant 




at p 


<.05, while 


differences greater than or equal 


to r=.33 




are 


significant 


at p<.01 


• 







Items are shown in Table 3. 



The higher the ability of the examinees, the more likely they were to 
report ^.hat both individual items and the whole test were too easy and that 
the items were appropriate to their ability levels. Higher ability examinees 
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also felt better about their levels of performance on the test in comparison 
to their performance on similar tests. Low-Ability examinees were more likely 
to report that the test items were too hard and that they had been frustrated 
by the difficulty of the test. Thus, Low-Ability examinees in general re- 
ported the test to be a difficult, inappropriate, and frustrating experience. 

A comparison of these correlations within the KR and No-KR groups shows 
that they are similar to each other in the direction of relationship. All 
values of r in the No-KR group, except that for Item 6, were significant at 
p<.001. The correlations under KR conditions were generally higher in absolute 
value than were those under No-KR conditions, thus indicating somewhat more 
accurate perceptions of item difficulty for examinees receiving KR. But in no 
case was the difference between the KR and No-KR correlations for a given item 
statistically significant. Thus, it is evident that examinees not receiving 
objective KR still were receiving moderately accurate subjective KR. 

All of the correlations within the conventional test group were signifi- 
cant at p<.001, as were those for all but item 5 and 6 within the stradaptive 
test group. The correlations for the conventional test group were uniformly 
and significantly higher than were those for the stradaptive test group. This 
indicates that students taking a conventional test were perceiving the degree 
of item difficulty in relationship to their ability level more accurately than 
were students taking the stradaptive test. 

The less accurate perceptions of the relative degree of item difficulty in 
the stradaptive test group are consistent with what would be expected given the 
adaptive process of item selection. That is, higher ability examinees were 
administered relatively difficult items, while lower ability individuals re- 
ceived relatively easy items. Thus the tendency for high ability examinees to 
rate items as relatively easy and for low ability examinees to rate them as 
relatively difficult was reduced substantially from that found in the conven- 
tional test. 

Reactions to the Provision of Knowledge of Results 

Responses to KR items . Table 11 summarizes the responses of the total 
group receiving KR to the items concerning their reactions to the provision of 
KR. There were no significant differences in these responses as a function of 
testing strategy. In general, responses were highly favorable. The modal 
responses indicated that KR: 1) made the test "much more interesting"; 2) in- 
terfered "not at all" with tes tees' ability to concentrate on the test; 3) did 
"not at all" make them nervous; and 4) caused them to try "somewhat" harder to 
get test questions correct. Examinees also indicated that they were bothered 
somewhat when their answers were incorrect, that they were "very interested" 
in knowing whether their answers were right or wrong, and that they liked get- 
ting KR. Thus, examinee reactions indicated that receiving KR made the test a 
more positive and interesting experience and that for most examinees KR did 
not have the adverse effects of making them nervous or of interfering with 
their ability to concentrate. 

Percentages of positive and negative KR . Table 12 shows the correlations 
between the percentage of correct responses (and therefore the percentage of 
positive KR) and reactions to the provision of KR. Four of the obtained cor- 

« . • If ^^.^^^..^^ •jm^t>»A <s4 am4-^4#»avi4-1 V -f f^oront* -from seiPO. 
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Table 11 
Response Percentages to Items 



Item and Response 








Category 


N 


% 


Mean S.D. 



1. Did getting feedback on this test 

make it more or less interesting? 3.71 .57 

Much more interesting 128 72.7 

Somewhat more interesting 38 21.6 

Didn^t make any difference 6 3.4 

Somewhat less interesting 2 1.1 

Much less interesting 2 1.1 

2. Did receiving feedback interfere with 

your ability to concentrate? 3.77 .55 

No, not at all 145 82.4 

Yesy somewhat 24 13.6 

Yes, moderately so 5 2.8 

Yes, very much so 2 1.1 

3. Did getting feedback make you 

nervous? 3.7 .59 

No, not at all 128 72.7 

Yes, somewhat 40 22.7 

Yes, moderately so 5 2.8 

Yes, very much so 2 1.1 

4. Did you try harder to get the 
question right because you knew* you 

would get feedback? 2.4 1.04 

No, not at all 32 18.8 

Yes, somewhat 71 41.8 

Yes, moderately so 29 17.1 

Yes, very much so 38 22.4 

5. How did you feel when your answers 

were incorrect? 2.4 .83 

It bothered me a lot 25 14.8 

It bothered me some 63 37.3 

It bothered me a little 69 40.8 

It didn't bother me at all 12 7.1 

6. Were you interested in knowing whether 

your answers were right or wrong? 3.7 .59 

I was very interested 132 75.9 

I was moderately interested 32 18.4 

I was somewhat interested 9 5.2 

I didn't care at all 1 .6 

7. How do you feel about getting feedback? .3.8 .67 

I'd rather not 10 5.8 ; 

I really don't care 8 4.6 

I liked getting feedback 155 89.6 

Note. Items are numbered 1 to 7 only for purposes of discussion. The order 
in which the items were administered Is shown in Table 4. 
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Examinees receiving greater proportions of positive KR were more likely to 
indicate that KR made the test more interesting (Item 1) and did not interfere 
with their ability to concentrate on the test (Item 2) . These examinees also 
indicated more interest in knowing whether their answers were right or wrong 
(Item 5) and were more likely to say that they liked getting KR (Item 7). 
There was no relationship between percentage of positive KR and the extent 
to idiich KR was reported to make examinees nervous, to cause them to try 
harder to get questions right, or to bother them when answers were incorrect. 



Table. 12 



Correlations Between Responses to KR Items 
and Percentage of Positive KR Received 


Item 


r 


1 


-.18** 


2 


-.17* 


3 


O.CO 


4 


-.01 


5 


-.20** 


6 


.06 


7 


.16* 


*Value of r significantly different from 



zero (p<.05) 
**Value of r significantly different from 
zero (p<.01) 

In order to determine if the quality of KR, whether computer-administered 
or subjectively perceived, differed between the conventional and stradaptive 
test, characteristics of the distributions of percentage correct scores within 
the two tests were examined. The mean percentage correct on the conventional 
test was .53, and that on the stradaptive test was .49. On both tests, then, 
KR was generally about half positive and half negative. 

However, there was more variability among people in the quality of KR 
on the conventional test. The standard deviation of percentage correct scores 
on the conventional test was .20. In contrast, the standard deviation of 
percentage correct scores on the stradaptive test was .08. Thus, the quality 
of KR on the stradaptive test was more constant across individuals than it 
was on the conventional test. 

SUMMARY AND CONCLUSIONS 

The results of this study indicate that both adaptive testing and the 
provision of immediate knowledge of results have significant effects on the 
psychological reactions of examinees to testing conditions. The ability level 
of testees was found to be related to their general reactions to testing. 
Ability level also moderated the effects of testing conditions on their psycho- 
logical reactions. 



Motivation 



In the total group of subjects, the correlation between reported 
motivation and the All-Item ability estimate was r«.21; this value, while 
indicating a relatively small degree of relationship, was significantly 
different from zero. In addition, results from tne analysis of variance of 
motivation scores indicated a significant main effect for Ability Group, 
with the High-Ability group reporting higher levels of motivation than the 
Low-Ability group. 

While High-Ability examinees generally reported higher levels of test- 
taking motivation than did Low-Ability examinees, this difference was moderated, 
and in some cases eliminated, by the differential impact of testing conditions 
on levels of reported motivation in the two groups. Specifically, testing 
conditions were found to have significant effects on the reported motivation 
of Low-Ability students but not on the ?30tivation of High-Ability students. 

High-Ability examinees reported high levels of motivation on both the 
conventional and stradaptive tests, and with KR and without KR. Low-Ability 
examinees, on the other hand, reported significantly higher levels of moti- 
vation on the stradaptive test than on the conventional test, although their 
levels of motivation did not differ as a function of KR conditions. Thus, 
while on the conventional test levels of motivation reported by High-Ability 
examinees were significantly greater than were those reported by Low-Ability 
examinees, on the stradaptive test the levels of motivation reported by the 
Low-Ability examinees were equal to or greater than those reported by the High- 
Ability group. 

These results indicate, therefore, that adaptive testing Increases the 
motivation of low-ability examinees, and consequently, their levels of 
motivation are comparable to those of high-ability examinees. The incentive 
effects of adaptive testing in the Low-Ability group also were reflected in 
their levels of test performance. Results reported previously (Betz & Weiss, 
1976) indicated that low-ability examinees obtained higher average scores 
on the stradaptive than on the conventional test. 

The findings that high-ability examinees were highly motivated regardless 
of testing conditions, while the motivation of low-ability examinees was 
significantly higher under certain conditions of test administration than under 
others are consistent with the hypotheses of Samuda (1975), Sweet and Ringness 
(1971), and others. These writers hypothesized that the inferior levels of 
performance of either lower-class or black individuals — in general, individuals 
who typically have not performed well on ability tests — may be due partially to 
lower levels of motivation. The results of the present study when considered 
in conjunction with those of Betz & Weiss (1976) indicate that adaptive testing 
can significantly increase the motivation of "low-ability" examinees and that 
the increased motivation is accompanied by sicmificant increases in ability 
test scores. 

Anxiety 

Significantly higher levels of anxiety were reported on the stradaptive 
test than on the conventional test. Examinees taking the stradaptive test were 
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more likely to report that they worried about how well they would do and that 
they were nervous and tense while taking the test than were examinees taking 
the conventional test. While there were no overall mean differences in 
anxiety as a function of KR conditions, more anxiety was reported on the 
stradaptive test, but less on the conventional test, when KR was provided. 
In both subject groups, the highest level of anxiety occurred in the 
stradaptive-KR condition, while the lowest level occurred in the conventional- 
KR condition. 

Interestingly, the condition which led to the most anxiety in Low-Ability 
examinees was also the condition leading to the highest levels of performance 
(Betz & Weiss, 1976) and high levels of motivation. Thus, adaptive testing 
in combination with the provision of knowledge of results increased test scores 
and motivation, but also led to a significant increase in reported anxiety. 

These results contradict the research evidence showing a negative 
relationship between anxiety and test performance (e.g., Sarason, 1958, 
1960, 1972; Wine, 1971). However, one possible explanation of these results 
may be found in the construct of facilitating anxiety, first postulated by 
Handler and Sarason (1932). Handler and Sarason theorized that anxiety has 
detrimental effects on performance only for individuals who have learned to 
respond to anxiety with a class of behaviors which interfere with performance; 
for such individuals, anxiety is "debilitating". For other individuals 
anxiety is "facilitating" because it elicits task-relevant responses. That is, 
an increase in the number of task-relevant responses may facilitate performance 
on the task. 

While debilitating and facilitating anxiety are postulated to be individual 
differences variables dependent on the individual's social learning history, 
it is possible that testing conditions may affect the extent to which "task- 
relevant" responses, as opposed to "interfering" responses, are elicited. Wine 
(1971) reviews research which indicates that situational conditions affect the 
extent to which "task-relevant" versus "self -relevant" responses are elicited; 
"self-relevant" or "self-focusing" responses are similar in effect to Handler 
and Sarason 's "interfering" responses. 

Although the results indicated that anxiety may have facilitated the 
performance of Low-Ability examinees taking the stradaptive test with KR, 
facilitating anxiety does not appear to be a general moderator of the effects 
of KR (or other testing conditions) Hon test performance. For High-Ability 
examinees, test scores increased with KR, but levels of anxiety did not 
differ significantly as a function of KR conditions. For Low-Ability examinees 
administered the conventional test, higher test scores but lower anxiety 
levels were observed under KR conditions (see Betz & Weiss, 1976). 

..-,1. 

As was the case with levels of motivation, the levels of anxiety reported 
by Low-Ability students were more influenced by modifications in testing conditions 
than were those of High-Ability students. For the High-Ability group, the 
difference between the mean anxiety scores from the most anxiety-producing 
situations (stradaptive-KR) to the least (conventional-KR) was .63. In the 
Low-Ability group, the mean difference corresponding to the same testing situar- 
tions was 1«47, a difference more than twice as large. 
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The findings that KR conditions did not have significant effects on anxiety 
are in contrast with those of Angell (1949), Bierbaim (1965), and Strang and 
Rust (1973) that examinees receiving KR reported more anxiety than did those not 
receiving KR. However, in these three studies, KR was provided via Pressey- 
punchboard devices or specially constructed answer sheets which required more 
effort from examinees receiving KR. Also, examinees in the studies of Angell 
and Bierbaum were required to continue responding until they found the correct 
answer; this procedure could have contributed to the increased nervousness re- 
ported by examinees receiving KR. Finally, the dependent variable in all three 
studies was performance on a classroom achievement test. Since the test scores 
of examinees were used to determine their course grades, receiving knowledge of 
incorrect answers may have increased the stress of the situation. 

In the present study, the provision of knowledge of results on a computer 
terminal did not require additional effort from the examineel, and the KR was 
given immediately following the first response to an item. Thus, within a com- 
puter-based testing environment, providing knowledge of results to examinees 
was found to lead to significantly higher levels of test performance (Betz & 
Weiss, 1976) without increasing levels of debilitating anxiety in the testing 
situation. 



Perception of Test Difficulty 

Results indicated that examinees in general reported the experimental teat 
(i.e., either the conventional or the stradaptive test administered with or 
without KR) to be a rather difficult and frustrating experience. They were more 
likely to consider test items as too difficult for them than too easy for them 
and felt that the experimental test as a whole was too difficult for someone of 
their own ability level. 

Reactions to four of the six "perception of difficulty" items varied signif- 
icantly as a function of KR conditions; examinees receiving KR perceived the test 
as a somewhat less difficult and frustrating experience than did examinees not 
receiving KR. In addition, there was some tendency for examinees taking the 
stradaptive test to perceive the test as more difficult for them and to perceive 
that they had done comparatively less well on the test than did examinees taking 
Che conventional test. This latter finding, however, is not surprising consid- 
ering the fact that about two-thirds of the sample consisted of 'high-ability 
students. These students probably are accustomed to doing relatively well on 
ability tests, and an adaptive test is designed to be relatively more difficult 
for such examinees than is a conventional test. 

Results also indicated that examinees' perceptions of the difficulty of 
the test in relationship to their ability level were moderately accurate whether 
or not they received computer-administered KR. That is, the students' subjective 
knowledge of results corresponded somewhat to their actual levels of performance 
on the test. For the total group of examinees, the absolute value of the cor- 
relations between ability level and responses to the four questions dealing 
specifically with the difficulty of the test Items ranged from .33 to .48. Thus, 
examinees who did relatively well on the test perceived the items as easier for 
them than did examinees who performed less well. 

32 
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Further, while the perceptions of examinees who received KR were slightly 
more accurate than were those of examinees who did not receive KR, In no case 
was there a statistically significant difference between the correlations of 
perception of difficulty and ability level In the KR and No-KR groups. In the 
former group, the absolute values of the four correlations ranged from .31 to 
.55, while In the latter group the values ranged between .32 and .42. 

These results support the hypotheses of Ammons (1956) and Ross (1933) that 
examinees always are receiving some subjective knowledge of results. That Is, 
examinees do have some Idea about how well they are doing on a test whether or 
not they are given KR. Ross posited the existence of subjective KR In order 
to explain the absence of effects In a study In which KR was provided to 
examinees, but the results of the present study Indicate that computer-adminis- 
tered KR can Increase test scores even though examinees are receiving moderately 
accurate subjective KR. 

Differences in the extent of the relationships between ability level and 
perception of test difficulty on the conventional and stradaptlve tests corres- 
ponded to those expected, given the differences between the two tests In the 
method of selecting the Items to be administered to a given Individual. On the 
conventional test, all examinees were administered a series of 50 Items selected 
to be most appropriate for Individuals of average ability level. Consequently, 
most Items were expected to be too easy for hlgh-ablllty students and too 
difficult for low-ablllty students. The absolute values of the correlations 
between ability level and responses to the perception of difficulty Items In 
the conventional test group ranged from .44 to .68. These values are similar 
to the correlation of .71 found between students* estimates of their scores and 
their actual scores in the study of Ross (1933). 

The stradaptlve testing strategy, on the other hand, is designed to select 
for administration the items which are most appropriate or closest to each 
testee's ability level. High-ability examinees are administered relatively 
difficult items, while low-ability examinees receive relatively easy items. In 
the stradaptlve test each examinee, regardless of ability level, converges on 
items with difficulties such that there is a 50% chance of correctly answering 
each of the items. While this ideal can only be approximated, the perceptions 
of examinees about how well they are doing on the test should not be as closely 
related to their ability levels as they are on a conventional test. 

The absolute values of the correlations between ability level and renponses 
to the test difficulty items in the stradaptlve group ranged from .19 tc 33. 
These values all were significantly lower than the corresponding values the 
conventional test group. While there was still some relationship between 
ability level and perception of performing well on the test, thii? relationship 
was far less pronounced than that found in the conventional test group. 

These results are consistent with the hypothesis that adaptive testing, as 
compared to conventional testing, has an incentive effect for low-ablllty 
examinees. If the results from the present study of convewtibnalt^ may 
be generalized to the situation of mgst group-adndnistered ability and aptitude 
tests, it is evident that low-ability individuals usually perceive themselves as 
doing relatively poorly on the test, whereas high ability examinees perceive 



( 



-29- 



themselves as doing well. On an adaptive test, however, the relatively low 
correlation between ability level and a testee's perception of the difficulty 
of the test indicated that Low-Ability examinees were perceiving themselves as 
doing only slightly less well than were High-Ability examinees. Thus, it may 
be assumed that the subjective KR received by Low-Ability examinees probably 
was more encouraging than that which they had been accustomed to receiving on 
conventional ability tests. Such feelings of encouragement may have contribu- 
ted to the higher test scores obtained by Low-Ability examinees taking the 
stradaptive test (Betz & Weiss, 1976). 

Reactions to the Provision of KR 

Reactions to the provision of knowledge of results generally were highly 
favorable. Most examinees reported that receiving KR made the test a more 
interesting experience and that they tried harder to do their best when they 
received KR. Few examinees reported that receiving KR made them nervous or 
interfered with their ability to concentrate on the test. The only possibly 
negative^_ef f ect of providing KR was that most testees reported being somewhat 
bothered when they found that their answers were incorrect. However, given the 
relative degree of accuracy of subjective knowledge of results even for examinees 
who did not receive KR, it is likely that examinees in both KR and No-KR condi- 
tions may have been somewhat discouraged when either perceiving or being told 
that they had answered an item incorrectly. 

In general, then, KR was evaluated favorably by the examinees, and it did 
not elicit adverse reactions which might have interfered with test performance. 
Overall, 90% of examinees liked receiving knowledge of results, and only 10% 
either did not like receiving KR or did not care whether they received it or 
not. These results correspond to those of Pressey (1950), who found that stu- 
dents reported that they liked receiving KR. 

As might be expected, there was some relationship between the quality of 
the KR received and the extent to which students evaluated KR favorably. Stu- 
dents who received higher percentages of positive KR were more likely to evalu- 
ate it favorably; small but significant correlations were found between the 
percentage of positive KR and responses to several of the KR items. Examinees 
receiving higher proportions of positive KR were somewhat more likely to report 
that they liked getting KR, that it made the test more interesting, that they 
were interested in knowing whether their answers were correct or incorrect, and 
that KR did not interfere with their ability to concentrate on the test. 

General Levels of Motivation and Anxiety 

Additional findings of the study concerned general levels of test-taking 
motivation and test anxiety. In general, this group of college students re- 
ported relatively high levels of motivation. Most examinees (88%) reported 
that they either frequently or almost always had tried to select the best answer 
to each question, and 65% reported that they were either fairly much or very 
much challenged to do their best on the test. Only about 14% thought that they 
could have done better if they had tried harder, and only 2% said that they 
didn't care how well they did on the test. Responses to only one item differed 
as a function of KR conditions; examinees receiving KR were more likely to 
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report that they could have done better if they had tried harder. However, 
given the small percentage of Individuals overall who selected this response 
alternative (1A%), reasons for the difference are not immediately apparent. 

While most examinees reported being motivated, students in general reported 
little anxiety in the testing situation. Modal item responses indicated that 
examinees worried somewhat about how well they would perform but did not feel 
particularly nervous or tense. The relative absence of feelings of nervousness 
or tension probably was due to the fact that test scores were not used to deter- 
mine course grades or to make any other decisions about the individual. There 
were no significant differences in responses to specific anxiety items as a 
function of KR conditions. 

Limits of Generallzability 

The results of the present study of the psychological effects of computer- 
assisted ability testing should be interpreted with caution for several reasons. 
First, with the exception of the research on test anxiety and state anxiety in 
testing situations, very little research effort has been directed at investi- 
gating the reactions of examinees to testing situations. For example, there was 
little previous research to provide guidelines about how to assess examinee 
motivation, subjectively perceived test difficulty, or reactions to the provision 
of KR. Accordingly, the items written for use in the present study must be 
regarded as a first attempt to assess a variety of examinee reactions to testing, 
and to assess psychological reactions to testing procedures. 

Second, because of the variety of reactions assessed, it was possible to 
include only a few items for each area -of interest. As a result, the motivation 
and anxiety scales were based on responses to only four and three items, respec- 
tively. More confidence could be placed in the results of future studies of the 
effects of KR on motivation and anxiety if such studies used scales that were 
based on responses to more than three or four items and that had been demonstrated 
to have adequate psychometric properties, e.g., internal consistency reliability. 
An obvious way to approach this problem is to use adaptive testing techniques, 
which permit the measurement of a large number of dependent variables in a minimum 
of testing time. 

It also should be realized that examinees may try to please the experimenter 
by responding in what they think would be the desired direction. However, this 
is a problem in all research of this type, and it would seem reasonable that if 
this tendency were present, it would have equal effect across experimental con- 
ditions. Thus, while total group descriptive data may be biased, any treatment 
differences found most likely would be a function of real differences in reac- 
tions. Furthermore, total group data concerning motivation, anxiety and the -.i 
other dependent variables from this study may provide a baseline for further 
studies of situational interventions in testing situations in similarly constituted 
populations and may be useful in comparative studies across populations. 

Finally, the results of this study and the previous study (fietz & Weiss,^^^^^ ^^^^^^^r ; ^^^^^ 
1976) are not presently generalizable beyond the college population used. Further: 
research of this type in groups of high school students and In other po^^^ 
would add considerably to the generallzability and potential utility of the^^-^^^^^^\^^^^^^^^ |^^ 
present findings. ■ / • 
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Concluslons 

The present study indicated that levels of test-taking motivation reported 
by low-ability examinees were significantly higher when they were administered 
an adaptive, rather than a conventional, test. High-ability examinees reported 
high levels of motivation regardless of testing conditions. 

AdaptLvR tasting, therefore, appears to increase the extent to which all 
examinees, rather than only high-ability examinees, are motivated to do well 
on an ability rest. The use of adaptive tests appears to result in comparable 
levels of motivation in examinees differing in ability level. Since the in- 
creased test-taking motivation of low-ability examinees taking the stradaptive 
test was accompanied by increases in mean test scores within this group, it may 
be concluded that adaptive testing creates conditions more conducive to allowing 
each individual to demonstrate his or her fullest capabilities in test performance. 

Significantly more anxiety was reported on the adaptive test than on the 
conventional test. But the conditions leading to highest reported levels of 
anxiety in the low-ability group (i.e., stradaptive with KR) were also the con- 
ditions under which they obtained the highest test scores and reported high 
levels of motivation. The construct of facilitative anxiety was offered as a 
possible explanation for this constellation of findings. 

Examinees, in general, had reasonably accurate perceptions of how well they 
were performing on the test whether or not they received KR. However, percep- 
tions of the relative degree of difficulty of the test were related much less 
closely to the ability level of the examinee on the adaptive test than they were 
on the conventional test. This suggests that adaptive testing creates a more 
equivalently reinforcing or encouraging environment for testees of all ability 
levels. In other words, adaptive testing may offer greater standardization of 
the psychological environment of testing. 

Finally, reactions to the provision of immediate knowledge of results were 
very favorable. Ninety percent of the examinees who had received KR liked re- 
ceiving it, and most indicated that receiving KR made testing a more interesting 
and positive experience. These results, along with the facilitative effects of 
KR on test performance (Betz and Weiss, 1976) suggest that further research, 
leading possibly to the wider implementation of immediate KR in testing proce- 
dures, is warranted. Thus, continued exploration and wider implementation of 
new approaches to measurement made possible by computer-assisted testing tech- 
nology seem critical, especially in light of the current concern with test fair- 
ness and bias. Such approaches not only may increase the accuracy of measuring 
instruments but also may begin to offset the growing public resistance to psy- 
chological testing. 
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